If not more, yeah. Okay. Okay. So the three of you and let's see m um myself and Maarika, who's not here, and but I think so it it occasionally Melissa will all be doing checking of the transcriptions. So this is the second pass these transcriptions are receiving and that's pretty straight-forward. I think you all kn basically have an idea what's involved. Um but the extra step that we're asking you to do f in this case is to look at the silent regions and listen to them for each channel and um make sure that no back-channels were lost for example, um yeah, back-channels is what you n typically would find, although by and large these are gonna be silent. So So you can um start s checking by y well you you know, you can order it in any way you like, but um start with the first channel and go through it entirely. Um I would recommend, I've found that it's easier when you're checking and you're trying to make sure the boundaries are right for a speech segment, I would recommend, like say this one, go down here to the uh trascription transcription tier and select that it so that uh when you play, you're certain that when it stops, you've caught the whole speech signal for that utterance and haven't lost any signal information. Um otherwise it it's liable to keep on running, and you might not notice where it where it finishes. So it it takes a little time. But um Alternatively, I you you could just listen to the speech segments again and make sure everything is correct for those, and so you could just hit um And then Like that. And then go back and check the the silence regions. How it if if you wanna make your job a little more interesting, you can d do it in m different ways on a different day. It doesn't matter. As long as it gets done. So Yeah, yeah, you're doing so that's that's the role of the checker is to check what's said in the silent regions uh in the speech regions and then make sure nothing was lost in the silent regions. You're also um responsible for resolving any um uncertainties that were marked by the original transcriber. And and you know, of course there are gonna be m many situations where you like the first transcriber can't make any sense of it. And in that case, it will be up to you to use the at symbol. Um and that can be used uh to represent one um unintelligible word or multiple intelligible words, but just use one at symbol. Um yeah, mm-hmm, exactly. And so that um once you go through all the s um speech regions and all the um non-speech regions, an o you could make another step of doing a search using um going under the edit menu you can do a search for, yeah, find replace for open parentheses and that'll take you to any that it any that the first transcriber might have marked out. So there's one. And then you can just resolve those all one by one as as a different set of um changes altogether. And and likewise for the double question marks. Just once you've gone through um and resolved all the open parentheses, you can do the same for op uh double question marks. And um and like I said, if you if you can't make any sense of it, then just use the at symbol. If you can um, just give it, you know, give it your best guess in these cases. If Mm-hmm. Yeah. Yeah, it's I would uh if you're v if you're quite certain, then I would go ahead and um type what you think it was said. If if you v think that n y there's maybe a th thirty percent chance you're wrong or or more Yeah. Right. So maybe a good idea would be to listen to each channel one by one, do listen to the silent and non-silent regions and make all those corrections, leave the unknowns, the um ambiguities, the uncertainties uncertainties until later. So go through all the channels and do that, and then um then you should have the context from all four speakers, and then go back and do those searches one by one for each channel. How would that be? Does that sound good? Okay. You'll you'll know all too well what was said in this meeting by the time you Yeah. Yeah, I think that would be really helpful. That would help us um, you know, nab a good number of those uncertainties. Mm. Choose one, it c I mean y think of it from a speech recognition point of view again. And always this should be this should motivate you. Yeah. What is most robust for a speech recognition and and leave out anything So you're well geared towards this task. Um So I've gone through Sorry. Oh no no no. Go ahead, go ahead. Too many. Okay okay. So um this is one I've gone through. I've just done one channel. And there were lots of lots and lots and lots of things as you can see. 'Cause I've got square brackets around all the things that need to be changed. Um so this person didn't ha um th obviously didn't know about capitalisation. So just about every utterance needs to be capitalised and and needs the end punctuation. Yeah. Um, no no no. Whatever um makes sense to you. Um but no, it it can continue into the next segment and that's perfectly fine. Um Yeah. Yeah. I'll review some of these things um after I go through this a bit um just to re-cap. Th these are things that I've after checking um t everybody's work um, these are things that were commonly left out or, you know misc mis-transcribed, whatever. So I'll go over those t as an opportunity to use the white-board. Um um okay. So uh this person left out a lot of end punctuation and um here's a case where I think she was transcribing an out-breath um or possibly an in-breath uh t uh in and used the hash mark. In any case, I didn't hear anything and and you shouldn't be transcribing in-breaths or out-breaths. That's n information we don't need. Oh, you know, this is the head-set mic and I heard something there. It was but it wa yeah, it was a bit of a yeah, it was creaky, so yeah f uh Yeah, well I didn't think it was im let's see, wait. Yeah. S there's there's actually quite a bit more in there than than I heard before. This is so I w I heard a creaky voice, and it was audible of the level of the rest of the speech. So I would I would keep that. I think that's good. Oh, okay. I was adding that, that's what I was doing. That makes sense. Okay, there you go. Um so there's things like that. Um sh there's a missing false start here um that didn't get caught. So I added that in. Um so that that's something that's quite common in the transcriptions that I've looked at. Missed false starts, um missed m Try to get it right. I mean the it I think syllables should be accounted for for th Yeah, well don't let it drive you crazy, 'cause there is one speaker Melissa just played for me today and he stutters horribly. I mean I I I'd you know, it's unfortunate for him and for us 'cause it's really gonna be a nightmare to transcribe. So you will have to draw the line somewhere. But if you c if it's say four repeated um if the word is repeated four times, uh then that's not too hard to count. I haven't come across something that's been I haven't come across anything exceeding four, I don't think. Have you? Mm-hmm. Mm-hmm. Uh-huh. Yeah, yeah. Yeah. Yeah. Make your best guess. Try to listen to it carefully enough, but not not so that you're obsessing over it for too long. I would say. Alright. Well there again, f uh think about robustness and speech uh recognition and it really doesn't matter. If it's a word fragment, who cares. It you're marking it as a fragment, and that's what's important. Well um I wouldn't de be using the star so liberally um for a non-native speaker. Mm-hmm. Well, right, right. Um I mean and and for that matter a lot of n native English speakers' realisations aren't going to be so true to um Yeah, yeah. Yeah. Well, I wouldn't bother to mark all of those instances though. No, I mean yeah, they're they're uh you're gonna find that for any speaker, they have a set way of of realising these things. And and it's just like Fiona was saying, um that the the word that stands out is something really uh aberrant. Like um I'm thinking of I I've used this example before, there's a guy who says project instead of product and uh it's something like project and it's probably just his his pronunciation of of. Yeah. Well, it it they were clearly talking about a product. So so uh I m I marked that as a m uh mispronunciation or s or something to flag anyway. Oh, evaluation. Yeah, I I've I remember that. That one I would I would flag, just like you did. It that's something definitely to mark up. Mm yeah. That's good. That's good. Alright. Yeah, I kn I don't know what to s tell you about n um certain speakers. I it's just something we have to deal with. Yeah. Yeah. Okay. So oop, I lost my lapel lapel mic. Yeah. Wait. Oops. Least I won't be transcribing the What do I care? Okay, so um there've there've been lots of missed filled pauses. Everybody seems to miss those, some more than others. So just really keep an ear out for those. Um Mm-hmm. Yeah. Yeah. I I'm not sure, I miss them plenty myself. But um this this second pass we should really try to catch all of them or at least most of th yeah. No, no. After. Okay. Yeah, it should be after every um acronym letter. Yeah, yeah. So at this point the um the only file that's been checked is Melissa one that Melissa did, and that was checked by me. Um so w we we're starting fresh with these and, you know, changes like that that were just announced we'll we'll be able to catch for these first passes. So yeah, y if you do see um something like um U_H_M_ for um, you you would change that to the regularised v version. Um and we just ch you know, this is arbitrary, but we had to choose something and not let it get too convoluted with um with choice. Here's Uh-huh. Mm-hmm, there. Um I Yeah, yeah. So it's not necessarily restarting the sentence, but just a restart of any type. A phrase. Yeah. Thank you for that description, Beata. Here's a uh so that answers your question about that answers your question about this one up here? Okay. Here's where I've I heard a another sound, another false start. Um or a stutter, I would say. Let me see. Oh yeah. I'm not seeing where this is. Okay. Okay, okay. Okay. So, that kind of thing. And there was a vowel in there, but you noticed that she just used the and that's fine. The the letter D_. That's she would like see. She Oh, sorry. I didn't go f back far enough. Yes. Um where is that. Yes, she's so she spelled it wrong. Um uh up here. Yeah, so I d I mentioned that because of spelling. And yes, any g I It's one word. So I d I just said spelling. It's an admonishment. Sorry? Uh like A_T_ is uh Mm-hmm. Yeah, yeah, yeah. If you have any questions about spellings, I i I think I m might have sent an email to this effect, just check the Oxford English Dictionary online. Uh and that's that will be our guide. Um, so a as far as I know, this is just missing an E_ cetera. Um but anyway. You can We've considered experimenting with running this through a spell check. You could take this as an Emacs file and run a spell check on it. But I think it would slow things down too much. So just hopefully i in going through this very carefully you're going to notice these misspellings. Mm-hmm. Mm-hmm. Yeah, yeah. Right. Yeah, I th I think you're right. So we'll we will just hope that you catch it. Um That's what we'll do. Okay. So Sorry. So I think she just she didn't catch the full form. Right, right. Yeah. She I think she meant to use the infinitive, but she just left off the to. A space between She would like see. She would like see. Yes yes yes, you're right. Yeah. Exactly. Uh that was what you were m remarking about. Okay. There we go. Let me save that. Okay. What else? Um be careful about um finding mis-transcriptions. So um So she t she transcribed that as would. And so Well, I think it's quite clear there. I mean obviously there are gonna be ca No, you try to keep it as true to what's being said. Then use um Mm-hmm. I'd have to hear it, because it m sounds like it could be a fragment. Uh it ju I'd just have to hear it. Um M I'd you don't want to b um be prescriptive and and type what you think they should've said. But but I I I do know the type of scenario you're describing. I just it's just hard to answer that without hearing something. Mm. Yeah, they should all I I stopped marking them, 'cause there are just too many. But Uh let's see. Oh yeah, y you're right. Right. That's right. Okay. There was a yes in there. Um and Yes. So another missed false start. Um There was creakiness on the she. But you don't need to mark that as a separate sound. Um Mm. If it sounds like um the laughter is just um interrupting a an utterance that continues after the laughter, I wouldn't worry about capitalising. If you hear just laughter on either side, yeah, do that. Yeah, yeah, mm I don't know, we'll have to I'll Okay, so laughter during speech. I'll f I'll talk to Melissa and Jean. S Mm-hmm. Mm-hmm. Mm-hmm. No, we don't have any qual tags like they use. But um I'll f I'll I'll figure that one out and put it up on the wiki or email you. Good question. I've come across that as well. Mm, see. Oh yeah, I don't know what that that noise was indicating. Um we can skip on. There are other types of things I wanna point out. Um let's see. Okay. Did you Did she say circuit? I didn't really hear that. So would would you have been tempted to put an asterisk after that? Uh yeah. Oh, okay. No, not necessarily. No, it's just something that the speech recognizer is liable to trip up on, I think. Mm-hmm, mm-hmm. Um y that may be, but I probably would flag it. I depending on how how different the vowel sound was. It's gonna be your call in a lot of these cases. Mm. Okay, here Here's a case of um where I found it n uh found a case of discontinuity. But you know, because she's a non-native English speaker, I don't I don't know. Maybe that was grammatically n uh unmarked for her. If you know what I mean. Maybe that wasn't uh her re-starting, maybe sh that was Mm mm-hmm, mm-hmm. So again I don't know. That w that was my reflex at the time to put that there. But Yeah. Well Unless you kinda get Right. Uh yeah, yeah. Yeah, I would type it as two. Mm okay, I'm gonna skip ahead. Oh, let's see, there was a c So I I hear she would quite clearly there and I think a good number of them you'll be able to resolve just just helps to have the context and be able to listen to it again. Well. Okay. Yes, it could be a lot of things. Um So there is there is another case where there wasn't a space between uh a non-word fragment and um a re-start. Okay. What else. There is an extra little symbol in here, and that was just kinda carelessness. Um So d even if it's something as brief as mm-hmm, we're we're gonna go ahead and capitalise that. And Maarika asked me what to do in the case of an utterance just consisting of laughter, do we do we use punctuation in these cases, and I don't know. I haven't been. I don't think it's necessary. Yeah. Uh no. Um Delete and collapse. I'm trying to underst I'm trying to interpret my note here. Delete and collapse the surrounding segments. Okay, I guess there there just really wasn't enough to transcribe there. Yeah. Yeah. So like I said, click down here just to if you haven't learned that trick already. And that won't play any surrounding segments. So there's nothing there worth transcribing. Um Uh Uh-huh. This one? Oh so you haven't put a hyphen there. I see, I see. Yeah. Okay. Commas. Commas did you say? Oh For g for that s type of situation? Yeah, okay. Commas are fine. I mean they're eventually they're going to be extracted globally, so it doesn't matter how you use the comma. Yeah, we can just wipe them out in one fell swoop. I I am too. I I like commas and then uh Steve Renals, who's project manager, had a look at m I think it was my transcriptions. What are all these commas doing in here? But he didn't he ultimately he said it wasn't a problem. They will I th they'll help in the case of discourse marking, but his point was that they're really subjective, I mean something that you the transcriber here Yeah, yeah. I really like using commas. Oh. Sh oh, she spelled cutest um with an I_, so that that's just something I pointed out. Okay, what else? Um So here's let's see. Now this was our So she typed uh our and I I didn't quite hear that. But But I you can see how it could've been, I don't know. Yeah, I don't know. Right. And you can never know, but if it fits well enough as a as an indefinite article, then you might as well. Oh, I was s I was suggesting that sh it be uh instead of our. But Yeah, I it's tricky, it gets ambiguous. Um I don't know. Uh what's this? Well oh, she's I hear the well there. The but this being a head-set mic channel m may make it easier to hear. She transcribed from the lapel mic. Uh Oh. Oh. Oh God. And you transcribed that speaker entirely? If that comes up again, we've got um headset mic files now. In fact, they replaced all the lapel mic files on the M_M_M_ server with head-set mic files, which we weren't asking them to do, but anyway they did it. And so now we don't have the lapel mic files. Apparently the headset mic files are are just a bit louder. And so you'll f probably find yourself turning the volume down considerably. But but apparently other than that they're just as good. But let us know, because we can retrieve the the other files. Yeah. Oh. Okay. I don't think I'm gonna go through uh any more of these, because I I think I've gotten through most of the types that I meant to point out. Uh right, right. Um what are we going to do for that. I d I did see s I've been looking at some guidelines for another corpus and I did see something about proper names, so I'll I'll have a look at that and see how we can handle that here. What Uh yeah, quite possibly, but since it's coming from ICSI uh it's coming from IDIAP in Switzerland, yeah, it w it should be easy enough to get, but it's it's one other step, you know, emailing somebody to find out who knows this information. Anyway, it's Yeah, yeah, because everyone signs a consent form when they participate in the experiment. So we ought to be able to get that. But but there shou we should also have a w a means of dealing with proper names when we don't know what it is or how it might be spelled at all and and flagging that in in some way. So I'll th I'll I'll try to address both things. Oh, just just type Mister fully. I M_I_S_T_E_R_, yeah. And same with missus, M_I_S_S_U_S_, I believe. Is it I believe I believe check check the dictionary. That yeah, but you would never um type the abbreviated spelling for anything. Um well, I'm sure there's a spelling in the O_E_D_, yeah. Uh-huh. Oh right. So the transcriber's enclosing everything in carets, and what you would do is That's a good question, because you will be leaving question marks there. Um okay. Right. Um it shouldn't, because things are going to be left like the asterisk will be left for the proc the subsequent processing. But carets with question marks in-between would have to be resolved in some way. Yeah. Yeah. Yeah. I I suppose that's all we can do really. You're not supposed to be polyglot transcribers here. Oh yeah yeah. If you if if f it's a foreign word and you know what it is, just enclose it in carets. Yeah. Yeah, yeah. If you if you're confident that you've got a fair grasp on French for example, why not. Yeah. Okay. Um there is there is just one other thing, she she typed okay um in caps. O_ c c O_ capital O_ capital K_ and huge no-no, I don't know why she missed that in the in the guidelines. Yeah. It is O_K_A_Y_. Yeah, yeah. Okay. Okay. Well, it will be your jobs to catch all of these things. So no um n no transcriber will be checking no checker will be checking their t own transcription. Yes. Yes. So we will catch these things more efficiently, I think. And on so I'll just go up to the board do I have to keep this stuff on? Oh yeah, they've got clips. Right, I'm mobile. Alright. Okay. So I guess just for the sake of using the white-board and killing a little more time I'll just write down the things that are most common to be checked. So let's see. So oh, okay. Now. Okay, s alright. Rather this is a list of things to do as the checker. So you wanna listen to the silent regions. Then check the speech regions. And as part of that you want to um check the boundaries. And oh, by the way, this should be um Um transcribe any missed speech. And just let's see what else. Okay. And within this checking the speech segments, you want to listen for or look for missed um Look for missed punctuation, filled pauses, words in stuttering. And if you have um if you have a stuttered word, you don't need to use the hyphen to mark discontinuity. It's just so if somebody says the the, then it's just the the. Um words in st uh And what else. Try to get f false starts. Short words, make sure with little words like the and uh, little articles and that kind of thing don't get omitted. Um short words and word fragments. Those are the commonly missed things. And then um look for cases of mis-transcription Seems so silly to be writing this all. But anyway, Mis-transcription, so make sure if the speaker said will, you type will not would or um if they say we, even though it might be more grammatically correct to type we're, um that kind of thing. Um tha cases of ellipsis, where um maybe the transcriber typed them, but really what was said was 'em, so where you use the apostrophe. And look for capitalisation I'm not gonna gonna do any more of this. Capitalisation. Spelling. Um, use the O_E_D_ to check on things you don't know about. Um refer to the wiki for all regularised spellings and f for things like filled pauses. Oh, was that W And then, yeah, capitalise the K_. Yeah. And I think smells really good. I used to always love smelling markers. What else did I find. Um the so make sure the da the dash, the hyphen, whatever you call it, is being used appropriately in cases of discontinuity, um but not in stuttering, like I said. And and don't use it when the utterance is clearly continued in the next segment or whatever. Um don't transcribe in-breaths or out-breaths. And okay should never be um two capital letters, it should be O_K_A_Y_. Um and one other thing is to break break up long segments if you find any. So Yeah, but that's a kind of crude guideline. I'd I'd say just anything that seems unmanageable. Ah. Yeah, yeah. Really uh it it means nothing to say a minute, it's just a g a really crude guideline. I would just try to keep segments very manageable um and a but u you know, use these wave-forms to help you. And you can adjust the resolution so that Oh. That didn't help me too much, but you can use this little slide bar to adjust the resolution. Actually that doesn't seem to be working. Which o Yeah. Right, okay, okay. So yeah. Mm-hmm. Oh okay, right. Uh-huh. Hmm? Ah right, that's good. Okay. That's excellent. Alright. Oh that's that's good to know. Okay. Yeah, so to help you make cuts, just look look f you know, use this wave-form and make sure you're not l losing any signal information. But um cutting segments down is a l a low priority, but it it will help ultimately, I think, post-processing wise. I don't Yeah, you don't don't get excessive about it. 'Cause that just ends up taking too much time. But I d I think from a speech recognition point of view it shouldn't matter, these segments being whatever length that they happen to be. I've seen other p people's transcription guidelines like f for the um ICSI project at Berkeley and elsewhere say indicating that it really doesn't matter and it's the point is just to get 'em into manageable units. And Yeah. Yeah, yeah. Yeah, that's true actually, that's the best reason for cutting them down, but hopefully the that'll be done during the first pass, because that's the whole point. Making the checkers' job a little easier. Um and then, yeah, just m um be sure after you've gone through the four channels to um do a search on all the unknowns, all the um cases of ambiguity or whatever. And what else was I gonna say? The way we'll be allocating this is is the same as before, th it's gonna be on the th the same the same page, just further down. Scroll down, you'll see w all the files. And uh I've already allocated some some things to be checked. So if you run out of work, go ahead and start checking, because this is something that's um considerably um bottle-necked at the moment. Yeah, I'd say for now um w we don't even have uh much new stuff to be uh to be gone over for a first pass. So it's important that we d um do this second pass. And I've indicated in a column whether something is priority. M I think I high typed high if it's high priority. Um so d those would be the ones you would focus on first, obviously. And when it come you don't want to change the file-name in any way. Just keep it the same. Um there there is going to be a checking in procedure. I don't know if anybody's mentioned this before, but you'll you'll know about this with the C_V_S_ server. So th Right, but it's be s y similar. But they're they're creating a their a guy Jonathan is creating a web form for you to do this all very simply and like just um upload everything that you've uh up upload the file that you've transcribed and and you'll be noting whether it's the first pass or the second pass and y your name um as the person who did that particular pass. And that will happen soon. I'll let you know. Yeah. Yeah, you'll have to I would suggest creating a f folder um called files to be checked or something like that. Yeah, and then within that create a s create sub-folders for the the given meeting and you'll have to get all the sound files unfortunately from the M_M_M_ server on your own, because the the AMI server's not very reliable for getting these sound files uncorrupted. So you'll have to get all the sound files from the M_M_M_ server and the person's transcription file should be in that folder where you're putting yours. Um and then you'll just uh yeah d you'll copy that over, you won't don't move it over, because we wanna have multiple versions of these things, but yeah, copy it into your directory and keep the same name. And then when you check it in, it will automatically be registered even though the the first pass was checked in when we're doing this checking in business, um it will know which pass it is, but I think you'll also be indicating that as well. Uh-huh. Mm-hmm. Yeah, it's it' actually six what is it? Is it three months, I think? Yeah. Yeah. They're they're three-month contracts and now, if you can't keep that, that's fine. But it's just a logistic thing with personnel where where we ha that was the minimum we were able to offer a contract for. So if you have to bow out, let us know as soon as you know when you you're gonna have to stop. Yeah, yeah, obviously those kinds of things are things we'll have to work around, but um yeah, it's just one of those details. Mm-hmm. I should think so. I mean the whole point w of keeping well, as long as you're able to, yeah, work out use of your computer with the other person, that's fine, yeah. The the whole point of having m regular hours was to make sure that somebody was here to answer your questions. But you've got keys, so so you should be able to get in during odd hours if if you wanted to work. Yeah, yeah. Yeah, I don't see why not, as long as you're, you know, doing your given set of hours for the week. Yeah. And um yeah. Paym I I think you'll all be paid. I think you'll all be paid uh well, have you not Mm. Okay. Okay. Really? No, that's certainly not necessary. Mm no. Well it If you have any th problems like that, mm um see Caroline Hastings. I mean if it's for this job and and getting paid, that that's not necessary and Caroline Hastings ought to be able to help you sort that out. But j also if if pay is a matter um like if it's a matter that you haven't gotten paid for some reason, um I hope that's not not the case. Mm-hmm. Talk to Caroline. Yeah, 'cause I've I really don't know um why that would be. I'm here full-time now, so I'll I'll be here m Monday through Friday and I l I leave at four now, so you won't find me anytime after that. But Yeah, my husband's at home with the baby. Yeah, yeah. He's uh four and half months. It's very lovely. A nice jolly little baby. Yeah, well um only as of Friday actually. So he was uh I was just telling Melissa this do you want us to stop? Yeah, yeah. Yeah yeah. Oh good, that worked out well.